Hypertext Transfer Protocol (HTTP)
Learn about the Hypertext Transfer Protocol (HTTP) and its importance in APIs.
Introduction#
The Hypertext Transfer Protocol (HTTP) is a stateless, application-layer protocol for distributed and hypermedia information systems. It's the foundation of data communication for the World Wide Web and is considered the de facto standard for client-server resource sharing. Web servers and client applications (browsers) must adhere to the message formats and transmission methods provided in the HTTP specifications. For instance, when we type a URL into a browser, the web server receives an HTTP request directing it to fetch and deliver the specified web page.
HTTP is based on the concepts of request and response, where one program (the client) requests another program (the server), and the server returns a response. The client usually makes requests in the form of HTTP commands, and the server responds with data in the form of documents. These commands are structured by different API architecture styles that we'll discuss in the coming chapters. Because it’s a driving force for APIs, HTTP is an essential protocol to understand.
As long as the request conforms to HTTP standards, the server will respond with the requested resources, regardless of the type of client.
This lesson discusses the essentials of HTTP. First, we explain its request and response formats. Next, we look into the HTTP methods, response status codes, and different HTTP headers. Toward the end, we discuss the significance of HTTP in APIs. Finally, we explain the secure version of HTTP, which is HTTPS.
HTTP message flow#
As discussed in the previous section, HTTP communication is initiated by the clients sending a request to the server, and the server responds. This section discusses the request and response message structure. The server parses each request to interpret the request and generate the response. In the same way, the client parses the received response from the server and collects the desired application data from the response message.
HTTP request structure#
An HTTP request message is composed of the following four components:
Method: HTTP provides built-in methods that determine what kind of action a client wants to request/perform.
The request-target and version: It can be a URL or URI, which is a unique address or identifier that determines a resource's location uniquely over the Internet. For example, the URL https://www.educative.io/courses/grokking-modern-system-design-interview-for-engineers-managers uniquely identifies an Educative course titled "Grokking Modern System Design Interview for Engineers & Managers." It also shows which version of HTTP is being used. The version part contains the HTTP version used for the request, for example,
HTTP/1.1orHTTP/2.0.Headers: HTTP headers allow clients to pass additional information between the communicating entities. These headers come in as a series of key-value pairs containing important information about each request. The request headers mainly contain the following information:
Server information from where data is requested.
Information about the browser being used by the user.
The kind of data formats a client can accept.
Body: The body part of the HTTP request is used to communicate any application data a client wants to communicate to the server. The format of data should be mentioned in the header so that the server clearly understands the data.
A sample HTTP request is shown below, representing all four parts of an HTTP request.
HTTP response structure#
The HTTP response has the same format as the request with the following modifications:
The method in the request is replaced with the HTTP version in response.
The URL in the request is replaced with a status code in response.
The HTTP version in the request is replaced with the phrase in response.
-
Status code and phrase: The status code is a three-digit number that conveys what happened with the client’s request on the server-side. The phrase part is a group of words that convey some meaning along with the status codes, for example, some of the phrases are
OK,Moved Permanently,Bad Request,Not Found, andHTTP Version Not Supported.The status codes are divided into the following five different categories.
- The status codes at one hundred levels (
100-199) are informational status codes. For example, the100shows that the initial part of the request by the client is received, and the client should continue. - The status code at the two hundred level (
200-299) represents that the client’s request was accepted successfully. - The status code at the three hundred level (
300-399) shows redirections. In other words, the clients must take some other actions to fulfill their requests successfully. - The status code at the four hundred level (
400-499) shows an error on the client-side. - The status code at the five hundred level (
500-599) represents an error on the server-side.
- The status codes at one hundred levels (
-
Headers: The headers in response provide extra information about the server and the response itself.
-
Body: The response body consists of the data from the server that a client has requested.
A sample HTTP response is shown below, including all three parts.
Note: Headers allow the exchange of extended control information between the client and server. There are a large number of headers available in the HTTP specification and they're extensively used.
Point to Ponder
Question
In HTTP, why are the control (headers) and application data included in the same request/response message?
There are several reasons that the control data should be unified with the application data in the same message, as discussed below:
- The HTTP is a stateless protocol that implies that the server should not keep any data relevant to a request. If we send control data in a separate message and application data in another message, the server should have to store the control data until the application data arrives. This approach diminishes the statelessness behavior of the HTTP protocol.
- Another reason is that it avoids sending too many requests, reducing the network’s resource consumption and data processing complexity on the client- and server-sides. Otherwise, we would be required to keep additional information both on the server- and client-sides.
HTTP methods#
The HTTP protocol is packed with several methods that are used by clients to request an action from the server. These methods are discussed below, along with respective curl commands to test them in the terminal on a publicly available domain, such as https://www.example.com.
Note: We should be aware of the fact that the names of these methods are case sensitive, so we should use upper case—for example,
HEADinstead ofhead.
The
GETmethod: This method is used to read data from a web source, for example, reading a page from the server. We can understand how a GET request works by using the following command:
The
POSTmethod: This method sends user-generated data to the server in order to create or update a data source. We can understand how thePOSTmethod works through the following command. Here, we pass a username,userA, and a password,SamplePassword, to thehttps://example.com/loginpage. However, in this case, it might give a404error (page not found) due to the unavailability of the login page.
The
PUTmethod: This method creates a new source or updates the existing one. It completely replaces whatever is present on the target URL. We can understand how thePUTmethod works by using the following command. Here, we’re updating the username toeducativeand email toedu@gamil.com.
The
DELETEmethod: This method deletes a resource from the server, for example, if we want to delete a user having a specific ID or name. In the following example, we delete a user having a username equal touser1:
The
TRACEmethod: This method echos the content of an incoming HTTP request. This method is used for debugging purposes. We can understand how theTRACEmethod works by using the following command:
The
CONNECTmethod: The connect method makes a two-way connection with a web server. It can be used to access a website using SSL or to open a tunnel. For instance, when a client wants to tunnel a TCP connection with a specific server (resource), they can ask the HTTP proxy server via theCONNECTmethod. The HTTP proxy server then establishes the connection for the client. We can try it with the following command:
The
OPTIONSmethod: This method queries options for a page supported by a web server. For example, we use theOPTIONSmethod if we want to see what HTTP methods a domain can support. Let's understand the working of theOPTIONSmethod in the following example:
The
HEADmethod: This method asks for information about the document. It’s faster than the other methods because significantly less amount of information is being transferred. We can try theHEADmethod in the terminal below using the following command:
We can try all the commands given above in the following terminal to see the working of each HTTP method.
We’ll see the following response to the HEAD method. The first line contains the HTTP version, the status code, and the phrase. The rest are some headers in the response.
The following table summarizes all the HTTP methods.
HTTP Methods and Their Purpose
HTTP Methods | Purpose and Features |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Note: HTTP also supports custom methods, which extend its functionality. For instance, WebDav provides a set of extensions to the HTTP protocol via custom methods such as
PRPFIND,PRPPATCH,MOVE,LOCK,UNLOCK, and so on. These custom methods are used for distributed authoring and versioning of documents.
HTTP headers#
As discussed earlier, the request to a server may contain additional lines specifying extra information, which are called request headers. For example, Authorization is a request header that represents a list of a client's credentials. Similarly, a response can also contain headers from the server-side, for example, Server is a header that shows information about the server in response. The nature of the headers in the request and response may vary, but in some cases, these may be applicable to both request and response. For example, the header Content-Type can be a part of both request and response. Following are some HTTP headers that can be used in requests, responses, or both.
Request and Response Headers
Header | Type | Description |
| Request | The type of pages the client can handle |
| Request | The time the resource was first downloaded from the server. It can be used to ask the server if a specific resource has changed since the client got it. |
| Request | A list of the client’s credentials |
| Response | Information about the server |
| Response | How the content is encoded or compressed |
| Response | The page’s length in bytes |
| Both | The protocol the sender wants to switch to |
| Both | Date and time of the message when it was sent |
| Both | Directives for how to treat caches |
The significance of HTTP in APIs#
The communication ecosystem was already present in the form of HTTP, and APIs used that ecosystem to their advantage instead of inventing something new. The adaptation of HTTP by APIs brings the following advantages:
The HTTP protocol is well-known to many developers, so it’s simple to use, upgrade, and scale.
It allows cross-platform and cross-language communication to support interoperability.
It provides a uniform interface for communication between entities in the form of request and response messages.
It’s a fast protocol because it requires minimal processing.
Due to its adaptability, APIs based on it can achieve a wide range of goals. For example, making custom requests based on the use case and utilizing different built-in methods provided by HTTP.
Repeat requests can be handled more quickly because responses can be cached.
HTTP supports various types of encoding techniques that are helpful in various types of communication with the server via APIs. The encoding techniques are carefully opted to make an API efficient.
Summary#
This lesson introduced a widely used client-server communication protocol known as HTTP. We described how an HTTP request and response are structured as well as learned about the commonly used HTTP methods. We also expanded on the different HTTP requests and response headers. Finally, we discussed the importance of HTTP in API design.
Quiz
Question
Which parts of the request and response messages are encrypted while using HTTPS?
HTTPS is HTTP over the Secure Sockets Layer (SSL); therefore, the entire HTTP request and response, including the headers and body, are encrypted while using HTTPS. Other details, such as the full domain or subdomain and the originating IP address, can be revealed via the DNS resolution and connection setup.
World Wide Web
Evolution of HTTP